Vehicle surroundings monitoring apparatus

ABSTRACT

A vehicle surroundings monitoring apparatus that extracts a body present in the surroundings of a vehicle as an object, based on an image captured by an infrared imaging device, including a binarized object extraction device that extracts a binarized object from image data obtained by binarizing a gray scale image of the image; a horizontal edge detection device that detects a horizontal edge in the gray scale image; a horizontal edge determination device that determines whether or not the horizontal edge detected by the horizontal edge detection device exists in a prescribed range from the top end or bottom end of the binarized object extracted by the binarized object extraction device; and an object type determination device that determines a type of object based on a determination result of the horizontal edge determination device.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a vehicle surroundings monitoringapparatus that extracts objects by performing binarization processing ofimages taken by infrared cameras.

Priority is claimed on Japanese Patent Application Publication No.2004-347817, filed Nov. 30, 2004, the content of which is incorporatedherein by reference.

2. Description of Related Art

Conventionally, a display processing device is known in which an objectsuch as a pedestrian with a possibility of colliding with a vehicle isextracted from an infrared image of a vehicles surroundings captured byan infrared camera, and information of this object is provided to thedriver (for example, see Japanese Unexamined Patent Application, FirstPublication No. H11-328364).

This display processing device searches a region (binarized object)where bright parts are concentrated by binarizing the infrared image,and determines whether or not the binarized object is a pedestrian'shead based on the distance calculated from an aspect ratio or fillingfactor of the binarized object, and also an actual area and a center ofgravity on the infrared image. Then, the height of the pedestrian on theinfrared image is calculated from the distance between the head partregion of the pedestrian and the infrared camera, and an average heightof adult persons, and a body region containing the body of a pedestrianis set. By separating and displaying the head region and body regionfrom other regions, visual aid for the driver is performed with respectto a pedestrian.

Incidentally, since the display processing device of the example of therelated art mentioned above detects a pedestrian based on determinationof the shape for a head part region or body part region on the infraredimage, it may become difficult to distinguish a pedestrian from amanmade structure that has a shape of a pedestrian, and particularly asimilar shape, size, and position height of the pedestrian's head andthat emits heat.

SUMMARY OF THE INVENTION

The present invention takes into consideration the abovementionedcircumstances, with an object of providing a vehicle surroundingsmonitoring apparatus that is capable of precisely distinguishing andextracting a pedestrian and a manmade structure on an infrared image.

In order to solve the above problem and achieve the related object, thepresent invention provides a vehicle surroundings monitoring apparatusthat extracts a body present in the surroundings of a vehicle as anobject, based on an image captured by an infrared imaging device,including a binarized object extraction device that extracts a binarizedobject from image data obtained by binarizing a gray scale images of theimage; a horizontal edge detection device that detects a horizontal edgein the gray scale image; a horizontal edge determination device thatdetermines whether or not the horizontal edge detected by the horizontaledge detection device exists in a prescribed range from top end orbottom end of the binarized object extracted by the binarized objectextraction device; and an object type determination device thatdetermines a type of object based on a determination result of thehorizontal edge determination device.

According to the vehicle surroundings monitoring apparatus describedabove, a manmade structure as a type of object in which horizontal edgesare difficult to detect in a prescribed range from top end or bottom endof the binarized object and something other than a manmade structure,for example a pedestrian having horizontal edges easily detected at thetop end or the bottom end of the binarized object, can be preciselydistinguished.

The vehicle surroundings monitoring apparatus may further include apedestrian recognition device that recognizes pedestrians present in thesurroundings of a vehicle based on the image, the pedestrian recognitiondevice executing pedestrian recognition processing on the object whenthe object is determined to be something other than a manmade structure,or to be a pedestrian, by the object type determination device.

In this case, pedestrian recognition accuracy can be improved byperforming the pedestrian recognition processing for the objectdetermined to be something other than a manmade structure as well as forthe object determined to be a pedestrian.

Furthermore, the vehicle surroundings monitoring apparatus may furtherinclude a warning output device that outputs a warning directed to theobject when the object is determined by the object type determinationdevice to be something other than a manmade structure or to be apedestrian.

In this case, since a warning can be output for an object determined tobe something other than a manmade structure as well as an objectdetermined to be a pedestrian, unnecessary warnings for a manmadestructure can be avoided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the construction of a vehiclesurroundings monitoring apparatus according to an embodiment of thepresent invention.

FIG. 2 is a diagram showing a vehicle equipped with the vehiclesurroundings monitoring apparatus shown in FIG. 1.

FIG. 3 is a flowchart showing the operations of the vehicle surroundingsmonitoring apparatus shown in FIG. 1.

FIG. 4 is a flowchart showing the warning determination processing shownin FIG. 3.

FIG. 5 is a diagram showing an example of a relative position between avehicle and an object.

FIG. 6 is a diagram showing an example of the classification of theregions such as an approach determination region set in front of thevehicle.

FIG. 7 is a diagram showing an example of a target region (mask) that isa horizontal edge detection target, being a region that includes abinarized object.

FIG. 8 is a diagram showing an example of a horizontal edge detected inthe mask shown in FIG. 7.

FIG. 9 is a diagram showing an example of the edge detection positionwhere the height position of the horizontal edge is a maximum (that is,the highest position in the vertical direction).

FIG. 10 is a diagram showing an example of the edge detection positionand the top end position of the binarized object.

FIG. 11 is a diagram showing an example of a target region (mask) thatis a horizontal edge detection target, being a region that includes atleast the lower portion of the binarized object.

FIG. 12 is a diagram showing an example of the horizontal edge detectedin the mask shown in FIG. 11.

FIG. 13 is a diagram showing an example of the edge detection positionand the bottom end position of the binarized object.

DETAILED DESCRIPTION OF THE INVENTION

Hereunder, a vehicle surroundings monitoring apparatus according to oneembodiment of the present invention is described with reference to thedrawings.

The vehicle surroundings monitoring apparatus according to the presentembodiment, for example as shown in FIG. 1, includes: an imageprocessing unit 1 equipped with a CPU (Central Processing Unit) thatcontrols the vehicle surroundings monitoring apparatus; two infraredcameras 2R and 2L that are capable of detecting distant infraredradiation; a yaw rate sensor 3 that detects the yaw rate of the vehicle;a vehicle speed sensor 4 that detects the traveling speed of thevehicle; a brake sensor 5 that detects a driver's braking operation; aspeaker 6; and a display device 7. For example, the image processingunit 1 detects a moving object such as a pedestrian or an animal infront of the vehicle in its traveling direction from infrared images ofthe surroundings of the vehicle that are captured by the two infraredcameras 2R and 2L, and from detection signals relating to the travelingstatus of the vehicle that are detected by each of the sensors 3, 4, and5. In the case where the possibility of a collision between the detectedmoving object and the vehicle is determined, a warning is output via thespeaker 6 or the display device 7.

Moreover, the display device 7 is, for example, constructed including adisplay device integrated with gauges that display various travelingstates of the vehicle, a display device such as a navigation device, andfurthermore an HUD (Head Up Display) 7 a that displays variousinformation at a position on the front window where the field of frontvision of the driver is not impaired.

In addition, the image processing unit 1 includes an A/D converter, thatconverts input analog signals to digital signals, an image memory, thatstores digitized image signals, a CPU (central processing unit), thatperforms various arithmetic processing, a RAM (Random Access Memory),that is used for storing data in the middle of the arithmeticprocessing, a ROM (Read Only Memory), that stores programs that areperformed by the CPU, and tables, maps and the like, and an outputcircuit that outputs drive signals for the speaker 6 and display signalsfor the HUD 7 a. The image-processing unit 1 is constructed such thatthe output signals of the infrared cameras 2R and 2L, and the respectivesensors, 3, 4, and 5 are input into the CPU after being converted todigital signals.

Furthermore, as shown in FIG. 2, two infrared cameras 2R and 2L aredisposed at the front of the vehicle 10 at positions symmetrical in thewidth direction relative to the central axis of the vehicle 10. Theoptical axes of both cameras 2R and 2L are parallel to each other, andboth infrared cameras 2R and 2L are secured at the same height from theroad surface. A characteristic of the infrared cameras 2R and 2L is thatthe output signal level (that is, luminance) increases as thetemperature of the object increases.

Moreover, the HUD 7 a is provided so as to display the images at aposition on the front window of the vehicle 10, where the field of frontvision of the driver is not impaired.

The vehicle surroundings monitoring apparatus according to the presentembodiment is provided with the construction described above. Next, theoperation of the vehicle surroundings monitoring apparatus is described,with reference to the drawings.

The operations in the image processing unit 1 for the detection of anobject such as a pedestrian, and outputting a warning, are describedbelow.

First of all, in step S1 shown in FIG. 3, the image processing unit 1obtains infrared images, which are the output signals from the infraredcameras 2R and 2L.

Next, in step S2, A/D conversion of the obtained images is performed.

Next, in step S3, a gray scale image containing half tone gradationinformation is obtained, and stored in an image memory. Here theinfrared camera 2R acquires the right image and the infrared camera 2Lacquires the left image. Furthermore, because in the right image and theleft image the horizontal position on the display screen for the sameobject appears displaced, this displacement (that is, parallax) enablescalculation of the distance from the vehicle 10 to the object.

Next in step S4, the right image obtained by the infrared camera 2R isassigned as the reference image, and binarization processing of thisimage signal is performed, that is, regions brighter than apredetermined luminance threshold value ITH are set to “1” (white), anddarker regions are set to “0” (black).

The processing of steps S4 through S9 above is executed for thereference image obtained from the binarization processing (for example,the right image).

Next, in step S5, the image data obtained as a result of binarizationprocessing for the infrared images is converted into run length data. Inthe run length data, regions that have become white as a result of thebinarization processing are displayed as lines at the pixel level. Eachline is set to have the width of one pixel in the y direction and thelength of an appropriate number of pixels in the x direction.

Next, in step S6, labeling of the object is performed for the image dataconverted into the run length data.

Next, in step S7, the object is extracted according to the labeling ofthe object. Here, in the case where lines including equal x directioncoordinates are adjacent to each other among respective lines of the runlength data, the adjacent lines are recognized to be forming a singleobject.

Next, in step S8, the center of gravity G of the extracted object, thearea S, and the aspect ratio ASPECT of the circumscribed quadrangles arecalculated.

Here, the areas S are calculated by adding the lengths (run (i)−1) ofeach run length data for the same object, assuming that the run lengthdata of the object labeled A is (x (i), y (i), run (i), A) (i=0, 1, 2, .. . , N−1; where N is an arbitrary nonnegative integer).

Furthermore, the coordinates (xc, yc) of the center of gravity G of theobject labeled A are calculated by multiplying the length (run (i)−1) ofeach run length data by the coordinates x (i) or y (i) (that is, (run(i)−1)×x (i) or (run (i)−1)×y (i)), adding the multiplication productsfor the same object, and dividing the result by the area S.

In addition, the aspect ratio ASPECT is calculated as the ratio Dy/Dx ofthe length Dy in the vertical direction of a quadrangle circumscribedaround the object labeled A against the length Dx in the horizontaldirection.

Since the run length data is represented by the number of pixels (numberof coordinates) (=run (i)), it is necessary to subtract 1 from theactual length (=run (i)−1). Furthermore, the coordinate of the center ofgravity G can be substituted for the center of gravity of thecircumscribed quadrangle of the object.

Next, the processing of step S9 and step S10, and the processing of stepS11 to step S13 are performed in parallel.

First of all, in step S9, time tracking of the object is performed, thatis, the same object is recognized in each sampling period. The timetracking is performed to determine whether two objects A and B extractedat time k, which is an analog time t discrete within a sampling period,are the same as the bodies C and D, which are extracted at the discretetime (k+1). When it is determined that the objects A and B are the sameas the objects C and D, the objects C and D are relabeled as objects Aand B. Then, the coordinates of each object that has been recognized(for example, the center of gravity) are stored in the memory as timeseries position data.

Next, in step S10, the vehicle speed VCAR detected by the vehicle speedsensor 4 and the yaw rate YR detected by the yaw rate sensor 3 areobtained, and by taking the time integral of the yaw rate YR, theturning angle θr of the vehicle 10 is calculated.

Meanwhile, in parallel to the processing in step S9 and step S10,processing for calculating the distance z between the object and thevehicle 10 is performed in step S11 through step S13. Because theprocessing of step S11 requires more time than that of step S9 and stepS10, it is performed for a longer period than steps S9 and S10 (a periodapproximately three times longer than the period for steps S1 throughS10, for example).

First, in step S11, one of a plurality of the objects tracked in thebinarized image data of the reference image (for example, the rightimage) is selected, and for example, the entire region enclosing theselected object by a circumscribed quadrangle is extracted as a searchimage R1 from the reference image (for example, the right image).

Next, in step S12, a search region for searching an image (correspondingimage) R2 corresponding to the search image R1 is set in the image (forexample, the left image) that corresponds to the reference image (forexample, the right image), and a correlation calculation is performed toextract a corresponding image R2. Here, for example, a search region isset in the left image according to the vertex coordinates of the searchimage R1, a luminance difference summation value C (a, b) which showsthe degree of correlation of the search image R1 in the search region iscalculated, and the region where this summation value C (a, b) is lowestis extracted as the corresponding image R2. Moreover, this correlationcalculation is performed for the gray scale image, not the image dataobtained from the binarization process. In the case where historicalposition data is available for the same object, the search region can benarrowed based on the historical position data.

Next, in step S13, the positions of the centers of gravity of both thesearch image R1 and the corresponding image R2, and the parallax Δd atpixel level are calculated. Furthermore, the distance between thevehicle 10 and the object that is the distance z (m), (object distance)from the infrared cameras 2R and 2L to the object, is for examplecalculated, based on the base length of the cameras that is thehorizontal distance D (m) between center positions of the each imagingdevice of the infrared cameras 2R and 2L, the focus distance of thecamera that is the focus distance f (m) of each lens of the infraredcameras 2R and 2L, the pixel pitch p (m/pixel), and parallax Δd (pixel),as shown in the numerical expression (1). $\begin{matrix}{z = \frac{f \times D}{\Delta\quad d \times p}} & (1)\end{matrix}$

Moreover, in step S14, when the calculation of the turning angle θr instep S10 and the calculation of the distance z in step S13 arecompleted, the coordinates in the image (x, y) and the distance z areconverted to real space coordinates (X, Y, Z).

Here, as shown in FIG. 2 for example, the real space coordinates (X, Y,Z) are set by appointing the center position of the mounting position ofthe infrared cameras 2R and 2L in front of the vehicle 10 as the originO, and the coordinates in the image are set so that the horizontaldirection is the x direction and the vertical direction the y direction,with the center of the image as the origin. Furthermore, the coordinates(xc, yc) are the coordinates that have been converted from thecoordinates (x, y) in the reference image (for example, the right image)into the coordinates in a virtual image obtained by aligning the originO of the real space and the center of the image data so that theycoincide, based on the relative position relationship between themounting position of the infrared camera 2R and the origin O of the realspace. $\begin{matrix}\left. \begin{matrix}{\begin{bmatrix}X \\Y \\Z\end{bmatrix} = \begin{bmatrix}{x\quad c \times {z/F}} \\{y\quad c \times {z/F}} \\z\end{bmatrix}} \\{F = \frac{f}{p}}\end{matrix} \right\} & (2)\end{matrix}$

Next, in step S15, a turning angle correction is performed to compensatefor the displacement of the object on the image caused by turning of thevehicle 10. This turning angle correction processing is to compensatefor displacement by Δx in the x direction in the range of the image datataken by the infrared cameras 2R and 2L, when the vehicle 10, forexample, turns to the left by an angle of θr within a period from time kto (k+1). As shown in the numerical expression (3) for example, thecompensated coordinates (Xr, Yr, Zr) obtained as a result ofcompensating the real space coordinates (X, Y, Z) are set as new realspace coordinates (X, Y, Z). $\begin{matrix}{\begin{bmatrix}{Xr} \\{Yr} \\{Zr}\end{bmatrix} = {\begin{bmatrix}{\cos\quad\theta\quad r} & 0 & {{- \sin}\quad\theta\quad r} \\0 & 1 & 0 \\{\sin\quad\theta\quad r} & 0 & {\cos\quad\theta\quad r}\end{bmatrix}\begin{bmatrix}X \\Y \\Z\end{bmatrix}}} & (3)\end{matrix}$

Next, in step S16, an approximated straight line LMV, which correspondsto the relative movement vector between the object and the vehicle 10,is calculated from N (N=approximately 10, for example) pieces of realspace position data constituting time series data, having undergoneturning angle correction, obtained for the same object during apredetermined monitoring period ΔT.

In this step S16, the most recent coordinates P (0)=(X (0), Y (0), Z(0)) and the coordinates P prior to sampling (prior to the predeterminedperiod ΔT) (N−1)=(X (N−1), Y (N−1), Z (N−1)) are corrected to thepositions on the approximated straight line LMV, and the correctedcoordinates Pv (0)=(Xv (0), Yv (0), Zv (0)) and Pv (N−1)=(Xv (N−1), Yv(N−1), Zv (N−1)) are calculated.

This procedure obtains the relative movement vector as a vector movingfrom the coordinates Pv (N−1) towards Pv (0).

By obtaining a relative movement vector by calculating an approximatedstraight line which approximates the relative movement track of theobject relative to the vehicle 10 from a plurality (for example, N)pieces of real space position data within the predetermined monitoringperiod ΔT, it is possible to estimate with better accuracy whether ornot there is a possibility of collision between the vehicle 10 and anobject, reducing the effect of position detection errors.

Next, in step S17, in the warning determination processing based on thepossibility of collision between the detected object and the vehicle 10,it is determined whether or not the detected object is subject towarning.

When the result of this determination is “NO”, the flow returns to stepS1, and the processing of step S1 to step S17 described above isrepeated.

On the other hand, when the result of this determination is “YES”, theflow proceeds to step S18.

Moreover, in step S18, in the warning output determination processcorresponding to whether or not the driver of the vehicle 10 isoperating the brake based on the output BR of the brake sensor 5, it isdetermined whether or not the warning output is required.

When the determination result in step S18 is “NO”, for example, in thecase where a degree of acceleration Gs (positive in the decelerationdirection) is greater than a predetermined threshold GTH while thedriver of the vehicle 10 is operating the brake, it is determined thatthe collision can be avoided by the brake operation, the flow returns tostep S1, and the processing of step S1 to step S18 described above isrepeated.

On the other hand, when the determination result in step S18 is “YES”,for example in the case where a degree of acceleration Gs (positive inthe deceleration direction) is not greater than the predeterminedthreshold GTH while the driver of the vehicle 10 is operating the brake,or in the case where the driver of the vehicle 10 is not operating thebrake, the possibility of collision is determined to be high and theflow proceeds to step S19.

The predetermined threshold value GTH is a value which corresponds toacceleration which would result in the vehicle 10 stopping after atraveling distance not greater than the distance Zv (0) between theobject and the vehicle 10 in the case where the degree of accelerationGs during the brake operation is maintained.

Then, in step S19, an audible sound warning is output, for example,through the speaker 6, or visual display warning is output, for example,through the display device 7, or tactual warning is output by generatinga fastening force that is tactually perceivable to the driver withgeneration of a predetermined tension to the seatbelt, or by generatingvibration (steering vibration), to a steering wheel for example, that istactually perceivable to the driver.

Next, in step S20, for example, the image data obtained from theinfrared camera 2R is output to the display device 7 to display therelatively approaching object as a highlighted image.

Hereunder, the warning determination processing in step S17 mentionedabove is described, with reference to the attached drawings.

This warning determination processing determines the possibility of acollision between the vehicle 10 and a detected object based on thecollision determination processing, processing to determine whether ornot an object is in an approach determination region, intrusioncollision determination processing, manmade structure determinationprocessing, and pedestrian determination processing, as shown in FIG. 4.The description below makes reference to an example as shown in FIG. 5,in which an object 20 is traveling at a velocity Vp in the direction ata substantially 90° angle relative to the traveling direction of thevehicle 10 (for example the Z direction).

First of all, in step S31 shown in FIG. 4, collision determinationprocessing is performed. This collision determination processingcalculates the relative velocity Vs of the vehicle 10 and the object 20in the Z direction in the case where, as in FIG. 5, the object 20approaches from a distance of Zv (N−1) to a distance of Zv (0) during atime period ΔT, and assuming that the heights of both the vehicle 10 andthe object 20 are not greater than a predetermined ground clearance Hand the relative velocity Vs is maintained, determines whether or notthe vehicle 10 and the object 20 will collide within the predeterminedtime allowance Ts.

When the determination result is “NO”, the flow proceeds to step S37that is described later.

On the other hand, when the result of this determination is “YES”, theflow proceeds to step S32.

Also, the time allowance Ts is intended to allow determination of thepossibility of a collision in advance of the estimated collision time bya predetermined length of time Ts, and is set to approximately 2 to 5seconds, for example. Furthermore, the predetermined ground clearance His set to approximately twice the height of the vehicle 10, for example.

Next, in step S32, whether or not the object is within an approachdetermination region is determined. As shown in FIG. 6 for example, in aregion AR0 which can be monitored by the infrared cameras 2R and 2L,this determination processing determines whether or not the object iswithin a region AR1, which is a distance (Vs×Ts) closer to the vehicle10 than a front position Z1, and which has a total width (α+2β) withpredetermined width β (for example approximately 50 to 100 cm) added toboth sides of the width α of the vehicle 10 in the vehicle lateraldirection (that is the X direction), and which has the predeterminedground clearance H; that is, an approach determination region AR1 wherethere is a high likelihood of a collision occurring with the vehicle 10if the object stays in that location.

When the determination result is “YES”, the flow proceeds to step S34that is described later.

On the other hand, when the result of this determination is “NO”, theflow proceeds to step S33.

Then in step S33, intrusion collision determination processing isperformed to determine whether or not there is a possibility of theobject entering the approach determination region and colliding with thevehicle 10. As shown in FIG. 6 for example, this intrusion collisiondetermination processing determines whether or not there is apossibility of the object in intrusion determination regions AR2 and AR3at the ground clearance H, where these regions are outside the approachdetermination region AR1 in the vehicle lateral direction (that is, thex direction), moving and entering the approach determination region AR1and colliding with the vehicle 10.

When the determination result is “YES”, the flow proceeds to step S36,which is described later.

On the other hand, when the determination result is “NO”, the flowproceeds to step S37, which is described later.

Then, in step S34, manmade structure determination processing isperformed to determine whether the object is a manmade structure or not.This manmade structure determination processing determines that theobject is a manmade structure and excludes the object from the warningdetermination if certain characteristics such as those mentioned beloware detected, meaning that the object cannot be a pedestrian.

When the result of this determination is “NO”, the flow proceeds to stepS35.

On the other hand, when the result of this determination is “YES”, theflow proceeds to step S37.

Then, in step S35, pedestrian determination processing is performed todetermine whether the object is a pedestrian or not.

When the result of the determination in step S35 is “YES”, the flowproceeds to step S36.

On the other hand, when the result of the determination in step S35 is“NO”, the flow proceeds to step S37, which is described later.

Then, in step S36, when in step S33 there is a possibility of the objectentering the approach determination region and colliding with thevehicle 10, (YES in step S33), or in step S35 the object determinedpossibly to be a pedestrian is not a manmade structure, (YES in stepS35), it is determined that there is a possibility of the vehicle 10colliding with the detected object and a warning is justified, and theprocessing is terminated.

In step S37, on the other hand, when in step S31 there is no possibilityof a collision between the vehicle 10 and the object within thepredetermined time allowance Ts, (NO in step S31), or in step S33 thereis no possibility of the object entering the approach determinationregion and colliding with the vehicle 10, (NO in step S33), or in stepS34 a determination is made that the object is a manmade structure, (YESin step S34), or when the object determined not to be a manmadestructure in step S34 is not a pedestrian, (NO in step S35), it isdetermined that there is no possibility of a collision between theobject and the vehicle 10 and a warning is not justified, and theprocessing is terminated.

Hereinafter, as the manmade structure determination processing in stepS34 mentioned above, processing to distinguish a manmade structurehaving a shape similar to a pedestrian, especially a similar shape andheight of a head that emits heat, is described.

As shown for example in FIG. 7, on a reference image (for example, theright image obtained from the infrared camera 2R), this manmadestructure determination processing sets a target region (mask) OA thatis a horizontal edge detection target, being a region that includes abinarized object OB.

For example, given the coordinates (xb, yb) at the top left point QL ofthe circumscribed quadrangle QB of the binarized object OB, a width Wbof the circumscribed quadrangle and a height Hb of the circumscribedquadrangle, if the width dxP of the mask OA is made a predeterminedvalue (for example, the width Wb of the circumscribed quadrangle+2×apredetermined value M_W; the predetermined value M_W being 1 pixel), andthe height dyP of the mask OA is made a predetermined value (forexample, 2×the predetermined value M_H; the predetermined value M_Hbeing a value expanding a predetermined height MASK_H of an actual space(for example, 30 cm) on an image, wherein the predetermined valueM_H=focal length f×predetermined height MASK_H/object distance z), thecoordinates (xP, yP) of the top left point AL of the mask OA are(xP=xb−M_W, yP=yb−M_H).

Then it is determined whether or not an edge filter output valueKido_buff obtained by applying an appropriate edge filter for detectinghorizontal edges to within the mask OA on the gray scale image isgreater than a predetermined threshold value KIDO_FIL_TH (for example,10 tones).

When the result of this determination is “YES”, the setting is made thata horizontal edge exists.

Meanwhile, when the result of this determination is “NO”, the setting ismade that a horizontal edge does not exist.

Then, as shown for example in FIG. 8, at each horizontal line I (I is aninteger, I=0, . . . , dyP−1) set in the mask OA, the ratio of the numberof pixels where the horizontal edge is determined to exist (edgedetection pixel number) to the total pixel number constituting thehorizontal line, that is, the inclusion ratio RATE (I) of the horizontaledge (=100×edge detection pixel number×(dxP−2×M_W)) is calculated.

Then, among the horizontal edges included in the horizontal line I wherethe inclusion ratio RATE (I) of the horizontal edge is greater than aprescribed threshold value RATE_TH (for example, 50%) (RATE(I)>RATE_TH), as shown for example in FIG. 9, the height position of thehorizontal edge where the height position of the horizontal edge is amaximum (that is the highest position in the vertical direction) is setas the edge detection position j_FL_TOP.

Then, it is determined whether or not the difference between the top endposition OBJ_TOP of the binarized object detected from the image dataconverted to run length data and the edge detection position j_FL_TOP(|OBJ_TOP−j_FL_TOP|) is greater than a prescribed threshold valueDIFF_H_TH.

The prescribed threshold value DIFF_H_TH is, for example, a valueexpanding a predetermined height H_TH (for example, 10 cm) of the actualspace on an image, wherein the prescribed threshold valueDIFF_H_TH=focal length f×predetermined height H_TH/object distance z.

When this determination result is “NO”, that is, when the top endposition OBJ_TOP of the binarized object and the edge detection positionj_FL_TOP are approximately the same position, then as shown in FIG. 10it is determined that the maximum height position of the horizontal edgewhere the decrease in the luminance value is relatively steep (edgedetection position j_FL_TOP) is detected at approximately the sameposition as the top end position OBJ_TOP of the binarized object, andthe binarized object is determined to be something other than a manmadestructure (for example, the head of a pedestrian) extending in thevertical direction (for example, upward in the vertical direction), andthe processing is terminated.

When this determination result is “YES”, that is, when the differencebetween the top end position OBJ_TOP of the binarized object and theedge detection position j_FL_TOP is relatively great, for example, thedecrease in the luminance value in the vertical direction is relativelymoderate, then it is determined that the binarized object is a manmadestructure extending in the vertical direction (for example, upward inthe vertical direction), and the processing is terminated.

That is, when the binarized object is determined to be a manmadestructure extending in the vertical direction (for example, upward inthe vertical direction) with the fluctuation in the luminance valuealong the vertical direction being relatively moderate, for example, ahorizontal edge is detected only near the peak position where theluminance value is a maximum, the horizontal edge becomes difficult todetect at the region where the luminance value changes to a decreasingtrend, and the difference between the top end position OBJ_TOP of thebinarized object and the edge detection position j_FL_TOP increases.

According to the vehicle surroundings monitoring apparatus describedabove, a manmade structure extending in a vertical direction (forexample, upward in the vertical direction) as a type of object in whicha horizontal edge is difficult to detect within the prescribed thresholdvalue DIFF_H_TH from the top end position OBJ_TOP of the binarizedobject and something other than a manmade structure, for example, apedestrian with a horizontal edge easily detected at the top endposition OBJ_TOP of the binarized object, can be preciselydistinguished.

In the embodiment described above, when the difference between top endposition OBJ_TOP of the binarized object and the edge detection positionj_FL_TOP is greater than the prescribed threshold value DIFF_H_TH, thebinarized object is determined to be a manmade structure extending inthe vertical direction (for example, upward in the vertical direction),but it is not limited thereto. For example, when the difference betweenthe bottom end position OBJ_BOT of the binarized object and the edgedetection position j_FL_BOT is greater than the prescribed thresholdvalue DIFF_H_TH, the binarized object may be determined to be a manmadestructure extending in the vertical direction (for example, downward inthe vertical direction).

That is, in the manmade structure determination processing, as shown inFIG. 11, on a reference image (for example, the right image obtainedfrom the infrared camera 2R), there is set a target region (mask) OAthat is a horizontal edge detection target, being a region that includesat least the lower portion of the binarized object OB.

For example, given the coordinates (xb, yb) at the top left point QL ofthe circumscribed quadrangle QB of the binarized object OB, a width Wbof the circumscribed quadrangle and a height Hb of the circumscribedquadrangle, if the width dxP of the mask OA is made a predeterminedvalue (for example, the width Wb of the circumscribed quadrangle+2×apredetermined value M_W; the predetermined value M_W being 1 pixel), andthe height dyP of the mask OA is made a predetermined value (forexample, 2×the predetermined value M_H; the predetermined value M_Hbeing a value expanding a predetermined height MASK_H of an actual space(for example, 30 cm) on an image, wherein the predetermined valueM_H=focal length f×predetermined height MASK_H/object distance z), thecoordinates (xP, yP) of the top left point AL of the mask OA are(xP=xb−M_W, yP=yb+Hb−M_H).

Then it is determined whether or not an edge filter output valueKido_buff obtained by applying an appropriate edge filter for detectinghorizontal edges to within the mask OA on the gray scale image isgreater than a predetermined threshold value KIDO_FIL_TH (for example,10 tones).

When the result of this determination is “YES”, the setting is made thata horizontal edge exists.

Meanwhile, when the result of this determination is “NO”, the setting ismade that a horizontal edge does not exist.

Then, as shown for example in FIG. 12, at each horizontal line I (I isan integer, I=0, . . . , dyP−1) set in the mask OA, the ratio of thenumber of pixels where the horizontal edge is determined to exist (edgedetection pixel number) to the total pixel number constituting thehorizontal line, that is, the inclusion ratio RATE (I) of the horizontaledge (=100×edge detection pixel number×(dxP−2×M_W)) is calculated.

Then, among the horizontal edges included in the horizontal line I wherethe inclusion ratio RATE (I) of the horizontal edge is greater than aprescribed threshold value RATE_TH (for example, 50%) (RATE(I)>RATE_TH), as shown for example in FIG. 9, the height position of thehorizontal edge where the height position of the horizontal edge is aminimum (that is the lowest position in the vertical direction) is setas the edge detection position j_FL_BOT.

Then, it is determined whether or not the difference between the bottomend position OBJ_BOT of the binarized object detected from the imagedata converted to run length data and the edge detection positionj_FL_BOT (|OBJ_BOT−j_FL_BOT|) is greater than a prescribed thresholdvalue DIFF_H_TH.

The prescribed threshold value DIFF_H_TH is, for example, a valueexpanding a predetermined height H_TH (for example, 10 cm) of the actualspace on an image, wherein the prescribed threshold valueDIFF_H_TH=focal length f×predetermined height H_TH/object distance z.

When this determination result is “NO”, that is, when the bottom endposition OBJ_BOT of the binarized object and the edge detection positionj_FL_BOT are approximately the same position, then as shown in FIG. 13it is determined that the minimum height position of the horizontal edgewhere the decrease in the luminance value is relatively steep (edgedetection position j_FL_BOT) is detected at approximately the sameposition as the bottom end position OBJ_BOT of the binarized object, andthe binarized object is determined to be something other than a manmadestructure (for example, the body of a pedestrian) extending in thevertical direction (for example, downward in the vertical direction),and the processing is terminated.

When this determination result is “YES”, that is, when the differencebetween the bottom end position OBJ_BOT of the binarized object and theedge detection position j_FL_BOT is relatively great, for example, thedecrease in the luminance value in the vertical direction is relativelymoderate, then it is determined that the binarized object is a manmadestructure extending in the vertical direction (for example, downward inthe vertical direction), and the processing is terminated.

That is, when the binarized object is determined to be a manmadestructure extending in the vertical direction (for example, downward inthe vertical direction) with the fluctuation in the luminance valuealong the vertical direction being relatively moderate, for example, ahorizontal edge is detected only near the peak position where theluminance value is a maximum, the horizontal edge becomes difficult todetect at the region where the luminance value changes to a decreasingtrend, and the difference between the bottom end position OBJ_BOT of thebinarized object and the edge detection position j_FL_BOT increases.

In this modification, a manmade structure extending in a verticaldirection (for example, downward in the vertical direction) as a type ofobject in which a horizontal edge is difficult to detect within theprescribed threshold value DIFF_H_TH from the bottom end positionOBJ_BOT of the binarized object and something other than a manmadestructure, for example, a pedestrian with a horizontal edge easilydetected at the bottom end position OBJ_BOT of the binarized object, canbe precisely distinguished.

While preferred embodiments of the invention have been described andillustrated above, it should be understood that these are exemplary ofthe invention and are not to be considered as limiting. Additions,omissions, substitutions, and other modifications can be made withoutdeparting from the spirit or scope of the present invention.Accordingly, the invention is not to be considered as being limited bythe foregoing description, and is only limited by the scope of theappended claims.

1. A vehicle surroundings monitoring apparatus that extracts a bodypresent in the surroundings of a vehicle as an object, based on an imagecaptured by an infrared imaging device, comprising: a binarized objectextraction device that extracts a binarized object from image dataobtained by binarizing a gray scale image of the image; a horizontaledge detection device that detects a horizontal edge in the gray scaleimage; a horizontal edge determination device that determines whether ornot the horizontal edge detected by the horizontal edge detection deviceexists in a prescribed range from top end or bottom end of the binarizedobject extracted by the binarized object extraction device; and anobject type determination device that determines a type of object basedon a determination result of the horizontal edge determination device.2. The vehicle surroundings monitoring apparatus according to claim 1,further comprising: a pedestrian recognition device that recognizespedestrians present in the surroundings of a vehicle based on the image,the pedestrian recognition device executing pedestrian recognitionprocessing on the object when the object is determined to be somethingother than a manmade structure, or to be a pedestrian, by the objecttype determination device.
 3. The vehicle surroundings monitoringapparatus according to claim 1, further comprising: a warning outputdevice that outputs a warning directed to the object when the object isdetermined by the object type determination device to be something otherthan a manmade structure or to be a pedestrian.